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FOREWORD 



As one of the ways to improve the quality of public education in the 
State, personnel in the. State Department of Public Instruction conduct an 
annual assessment of educational performance. This assessment provides 
educational decision makers with accurate and objective information for 
planning and administering the State's public elementary and secondary 



This year, a series of reports will be released on the performance 
of third-grade students. The reports will include reading, mathematics, 
language arts,-$ocial studies, science, cultural arts, health, and 
physical education. Also, special surveys on teachers' and principals 
opiniotis of education will be released. All of this information should 
also help the general public to be better informed about the status of 
their schools on a statewide basis. 

Aware of the fact that patrons and educators at the local school 
level also wish to know more about the quality of education in their 
/ Schools, the State Department of Public Instruction is initiating a 
program to assist local school personnel to conduct assessment programs. 
Constructive use of this information, as well as statewide data, will 
insure continuing progress in providing appropriate learning experiences 
for all children and youth in North Carolina. 



schools. 
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PREFACE 

As part of his total effort to initiate bettfer management techniques, 
the State SuperinteneTent of Public Instruction indicated in 1970 that 
more and better information was needed for state-level planning. He 
initiated the State Assessment of Educational Progress in response 
to that need. . 

The assessment program was a collaborative effort from the beginning. 
Many levels of the education community contributed suggestions. Funds and 
services for the program were obtained from local, state, and federal 
sources. Cooperation among local and state components of the public school 
system and the nationally respected Research Triangle institute Was 
the backbone of the assessment. There was an open exchange of ideas, 
experiences, and services. ^ 

As a result of these cooperative relationships, the first State 
Assessment of Educational Progress took place 1n the spring of 1972 
with mir/imal disruption to school programs. A statewide sample of sixth 
graders /participated by completing exercises in reading, mathematics, 
language arts, career awareness, and several dimensions of student 
attitudl^s- ' . 

At the recommendation of the State Board of Education, the 1973 
Legislature voted to fund the assessment program annually as part of 
the budget of the State Superintendent of Public Instruction. Concurrently, 
an advisory committee of legislators, businessmen, students, parents, and 
educators was formed to assist the State Boar^d and the State Department 
of Public Instruction on aspects of statewide assessment and accountability. 

A three-year cycle of assessment in grades three, six, and nine was 
established, beginning in 1974 with the State Assessment at the third 
grade. In the 1974 assessment, information was collected from teachers 
and principals as well as students. Student performance measures were 
taken in language arts, mathematics, cultural arts, reading, science, 
social studies, health, and physical education. Reports are now being 
prepared on the results. 
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INTRODUCTION 



Purpose of the Assessment 
In order to improve educational planning and decision -making in North 
Carolina, the' State Department of Public Instruction initiated a statewide 
assessment of educations] progress. The information gathered through this 
assessment operation has three basic purposes: 

1) To examine the state's present educational position: Knowing 
the educational status will enable educators to plan ^)etter 
programs for Improving" learning and teaching. Objective 
information will help decision-makers set program priorities 
with more assurance and allocate resources on the basis of 
need. An accurate description of the current status will 
increase general public knowledge and understanding about 

— the public schools. 

2) To measure educational progress over a \ period of ^ears: 
As time passes, a charting of the educcitional progress in 
this state can be made. These benchmarks of educational 
quality could become the basis for educational accountability 
for the state. 

3) To seek means of improving North Carolina's education: As 
more information is collected in the statelbssessments, 
variables which affect learning can be examined, and those 
variables which show a positive influence on learning can be 
promoted. 

In addition, the state assessment effort provides local units with 
technic!^^ assistance in planning similar local programs which aid the setting 
of l&al priorities. Goals may be set using meaningful state and regional 
norms which are made available from the statewide effort.' Furthermore, 
assessment information collected in each school will assist teachers in 
planning better instructional progran^ for public school youngsters and 
help patrons and parents to better understand the educational needs and 
attainments of North Carolina children. 



1 



ERIC 

hriimiimrmmia 



In a continuing attempt to develop and improve North Carolina's 
assessment program, the Legislature, adding its encouragement through program 
sup^rt, approved funds in 1973 as a part of the State Superintendent's 
program budget to underwrite the statewide assessment effort. This State 
Assessment at the third-grade level is the first stage in a proposed three- 
year assessment cycle. In 1974-75, assessment will occur in the sixth grade, 
and, in 1975-76, at the ninth-grade level. 

Implementation of the Assessment 

\ - " ■ 

^ The Sample \ 

Selecting third graders to participate in|the assessment program was 
the responsibility of the Research Triangle Institute, assisted by the 
State Department of Public Instruction. The objective was t^ choose a 
representative sample of size sufficient to provide reliable estimates of \ 
test score averages for the State, the Coastal Plains, the- Piedmont, and j 
the Mountains. Independent samples of 2500 students were considered 
appropriate for each of the five areas described^ in the section entitled 
"Assessment Areas". The total thircl grade enrollment of the eighteen 
schools containing 1970-71 state-supported kindergartens provided approxi- 
mately 2,000 students for a special assessment follow-up. 

A two-stage sampling procedure was designed to select the 12,500 
students for-the first five areas. In order to give each third grade 
student in North Carolina an equal chance of being chos^^, 618 schools were 
randomly selected with the probability of school selection based upoa 
stratification according to the size of the third-grade enrdllment. 

Random selection of students within schools was controlled to preserve 
the proportion of ESEA Title I enrollment within the third-grade class. 
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Of the 93J52 third-grade students in North Carolina, the chance of selection 
for any child was ten out of seventy-four. 

Field Procedures 

An Assessment Coordinator was designated by the superintendent of each 
participating LEA to organize all lassessment activities. The activities 
included: (1) selecting and coordinatiril^the testing schedule, (2) distrib- 
uting and collecting test packages and questionnaires, and (3) providing 

/ . 

Information and assistance'to the test administrators and princi^ls, 
With the approval of the;super*ntendent. Assessment Coordinators also selected 
someone other than the student's classroom teacher to administer the tests. 
These administrators read aloud all i^ems which did not test the student's ability 
to read. To insure standardization of test procedures, the Division of Research 
staff held workshops to acquaint coordinajiors and administrators with assess- 
ment procedures. 

« 

Assessment Areas 

The 1973-74 Assessment of Third Graders consisted of five different 
assessment areas and an additional research package for the evaluation of 
third graders who had previously attended state-supported kindergarten. 
In addition to student measiit:es, all teachers (grades K-6) and principals^ 
of the 618^schools included in the student sample were asked to Respond to 
questionnaires designed to reflect theii opinions about the educational 
needs and priorities in North Carolina. 

The subjects included in the six assessment areas and the type of 
testing involved are listed in Table 1. ^ 
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TABLE 1 ' 

OVERVIEW OF. 1973-74 ASSESSMENT AREAS . TESTING, AND SAMPLING 



Assessment Area 



Reading, Math, Language Arts 



leading. Math;, Langua9e Arts 
lealtj) and Physical EducatiorL 



CuTtijral 



Arts 

Science and Social Studies 

Third-Grade Kindergarten 
Follow-up ; 



Type of Testing 



Number of 
Students Sampled 



Norm Referenced 
(Iowa Tests of Basic 
Skills) 

Objective Based 

Objective Based; 

Motor Performance 
♦ 

Perception Survey 
Objective Based 



Norm Referenced 
(Iowa Tests of Basic Skill 
(Cognitive Abilities Test) 
(Self Observation Scale) 



2,500 
2,500 

2,500 
2,500 
2,500 



2,000 



Types of Instruments 
Reading, Language Arts, and Math were ^ach assessed by both a norm- 
o referenced tesX (iowa Tests of Basic Skills) and an objective based test 
developed at/the state level. The difference in the kinds of information 
provided by the two types of measurements should be considered when 
interpreting test results. ' . 

Nationally standardized achievement tests, such as the Iowa Tests of 
Basic Skills, are designed to provide information about student performance 

in given subject areas in relation to tfie^performance of jother students 

\ . I 

' who are representative of the nation as a whole.^ The national sample of 

"■^ i 

dents taking the ITBS is the "norm" or reference group to whose perform- 
ance we compare our state results. Thus, the ITBS provi(les/informal;ton-jjn^ 

the educational status of North Carolina third-grade students in relation 
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to'the performance of a natToRa+ -sample of "typical" third graders. Such 
standardized tests also assume a continuum of achievement skills based upon 
the scores of the national sample. North Carolina's third-grade results may 
be considered against this continuum. 

Norm- referenced tests are designed to spread out developmental scores 
on a continuum of skills spanning several grade levels. However, they 
do not tell us specifically what our students have achieved or how they 
perform on a given set of educational tasks. Some items on the ITBS can 
admittedly be grouped in^o^ subject area objectives, but the test is not 
designed for diagnostic" purposes. 

Therefore, objective- based tests were developed for the areas of 
reading, language arts, and mathematics in order to assess more specific 
knowledge of North Carolina's students. Program area specialists and 
researchers collaborated on this review and selection process. Question- 
naires were developed, informaiiort gathered, standardized tests carefully 
reviewed, and objectives and items (finally selected in accord with some 
of the major educational goals of North-^^rol ina. ^ 

Objective-based tests, also known as criterion-referenced tests, 
are developed differently from norm- referenced tests. They facilitate 
assessing the_ extent to which students have learned some defined behavior 
domain or specific class of learner skills. These behavior domains are 
also referred to as objectives. Specific obje^rtives considered important 

or crucial for later skills are selected for each subject area. Then, 

/ 

items selectejd-tS^ measure these objectives determine how well students have 
learned the knowledge or behavior described by the objectives. Objective- 
based' tests are thus diagnostic of specific learning, rather than more 

broadly comparative in nature - as are the norm-referenced tests. 

5 



strengths and weaknesses of a group of students for a given su"bject area 
are thus determined, and sometimes, though not necessarily., in relation to 
a norm group. \ 

It is important in making eduqational program decisions to know 
specifically what students have learned a$ well as how they are generally 
performing in relation to other students. For this reason, the assessment 
of third graders included experiinental objective-based tests for various 
subject areas. Norm-referenced and objective-based tests when combined, 
should provide a more complete picture of the performance of North Carolina 
students. 

In addition to student performance tests, other instruments were used 
in the North Carolina assessment. Tests were developed on student perceptionj 
in some subject areas, and a survey of teachers' and principals* needs was 
taken. The assessment staff also acquired school and community information 
'on variables known to be associated with achievement. 

Interpreting Test Scores 

Nomi-Referenced Tests 
. The knowledge that a student answered seventy-five items correctly 
on a ninety-item test tells little in itself about the achievement level 
of the student. If we know, however, that ninety percent of the students 
in the standardization sample earned scores lower than 75, we might conclude 
that the student in question performed rather well. The value assigned to 
a score, then, is determined by comparing -that score with scores earned by 
nriOTbers of the appropriate nom group. This process of comparing a student'^ 
score with a scale based on the test performance of the norm group gives 
useful, relative meaning to the individual student's score. 
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Systems have been developed for calibrating the distribution of norm 
group scores, making the comparison process easier and at the same time 
more informative. These systems clarify position of scores in relation 
to one another and deal with the problems of direction, distance 
and degree. However, as educational statisticians at the Research 
Triangle Institute have pointed out,..^^^"no single statistic exists for 
completely meaningful interpretation of the degree of difference between 
(groups of scores)." Therefore, these reports will include a variety of 
reporting systems to aid in the perception, of degree. 

One system is based on th$ relationship between the average number of 
items answered correctly by groups from successive grades (grade equivalent). 
Another compares a student's performance against the percentage of students 
in the national norm group whose scores fell below the student's score 
(percentile rank). Still another system looks at item performance. (Norms 
for item performance are available for the ITBS.) The percentage of students 
in the national norm group who answered each^ijem correctly provides a means 
of comparison for the item performance of North Carolina's students (item 
difficulty)- Other systems compare the total distributions of the North 
Carolina group against the total distribution of scores in the national 
norm group. 

The procedure for establishing national^ norms for comparison involves 
choosing a "representative" national sample of students, administering the y 
tests to them, and determining the distribution of their scores within each^l 
grade level. For example, if a median (average) vocabulary raw score of ; 
18 was attained by students tested during the first month of their third- / 
grade enrollment, the developmental concept grade equivalent (GE) would 
assign a score of 31,0 to the vocabulary raw score of 18. 



I other grade equivalent scores are established from the median (average) 

I 
I 

raw score attained by students at the beginning of other grades (i.e., 21.0, 

31.0, 41.0, etc.) on this test. Grade equivalent scores corresponding to 

leach of the ten months of school development between grades (31.0 to 41.0, 

etc.) are determined by dividing the raw scores between the reference points 

into ten intervals for the months of the school year and summer. These 

ten points, of course, represent an average year's development. It would 

be unreasonable to expect below average students to obtain ten points in 

a year while talented students should obtain more. The estimate of 

developmental skill from such a system is helpful. However, it does have 

limitations, is often misinterpreted and misused, and, therefore, generally 

not recommended as the only reporting device (see Appendix A of the 1973 

Assessment Report), The following paraphrased excerpt from the Teacher's 

^Guide to the ITBS is informative. 

The grade equivalent is an estimate of where the pupil is 
along the developmental continuum measured by these items, not 
of where he. should be placed in the graded organization of the 
school c A second grader with a grade equivalent score of 45 is 
at the 90th percentile of the second grade norm group, meaning 
that 90 percent of the second grade pupils scored lower and 
10 percent scored as well or better. This pupil should be 
considered as being in the upper 10 percent of the second grade . 
His grade equivalent of 45 does-^ - not indicate that he is ready 
for fourth-grade work or that he should skip the third grade. 

The publisher points out this limitation does not mean that grade 

equivalents should not be used at all. He continues, "They are valuable 

indicators of pupil growth [particularly for those not considerably above 

or below average] but should not be used to determine a^pupiTs standing in 

his grade Percentile norms and stanines are provided for this 

purpose." We concur and believe that looking at performance^' n several 

ways, while remembering the limitations of each , is the better approach to 

valid interpretation. 
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Objective-Based Tests 



Generally, objective-based tests results are interpreted by looking at 
the percentage of items achieved (or answered correctly) for a given objective. 
The desired level of achievanent for an objective is a considered, yet 
subjective decision on the part of educators. In some cases, 50 percent ^ 
achievement of an objective at that grade level may be acceptable; in others, 
100 percent may be considered necessary for acceptable performance. The 
leveKdepends on both the purposes for assessing the objective and whether 



or not the objective has been previously taught. In the statewide survey; 

/ 

objectives were selected that appeared to be coinnonly relevant to the 

\ 

curricular area throughout the state or that had some importance for 
state-level planning. Acceptable achievement levels may therefore vary 
with different subjects and objectives. This same process could be repeated 
at the regional or local level and the final test may again have different 
objectives, depending on local priorities. A statewide sample of outstanding 
third-grade teachers reviewed the s^ate selection of objectives this past 
summer for relevance and importance to their classes. They also examined 
the items and estimated the success they felt their students would achieve 
on them. However, due to possible differences which exist across the state, 
a "desired" achievement level was not set for North Carolina. 

Another consideration is the number of items per objective. As mentioned 
earlier in the "Types of Instruments" section, objectives reflect specific 
areas or domains of student behaviors. Because only a limited number of 
items can be selected for a given group of behaviors (objectives) the results 
on these items should be carefully interpreted as "indicators" of general 
pei^formance for the objective. 




If there are only two items per objective, the possible achievement 
levels for the objective are necessarily 0 percent, 50 percent, and 100 

percent achievement. Similarly with four items for the same objective, 

I 

! 

the possible achievement levels would be expanded to 0 percent, 25 percent, 
50 percent, 75 percent,- and 100 percent- This increase in items enables 
a more specific reporting of results^at both the student and group level- 
Further, we have a greater assurance that the students (or groups)^ have 
adequately learned the skills or beha^viors stated in the objective if four 
items are used rather than two- However, no set rules can be given, as 
some objectives can be stated with greater specificity than others and require 
fewer items for adequate measurement. 

Generally, statewide results will be repdrted by objectives and reflect 
the percentage of students who answer a given number of items correctly for 
the objective- For example, if an objective has three items, results will 
show what percentage of students answer one , two, three, or no^ items 
correctly- The number of items a student or group is expected to answer 
correctly is again a considered judgment based on the particular objective 
and the value or priority the user places on that objective. 

Information at the item level also aids in interpretation as efforts 
are made to diagnose specific strengths and weaknesses- Thus, the objective- 
based test allows flexibility to curriculum specialists in assessing important 
developmental skills with greater accuracy- Certainly it adds a valuable 
perspective to those who plan North Carolina programs to meet\ specific 
needs of North Carolina youngs ters- 
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Making Valid Conparisons of Test Scores in North Carolina 
Comparisons of test scores may be less than useful if the individuals - 
within the groups vary greatly on important characteristics. Comparisons 
of .the test scores of classes, schools, local school districts, or states 
must be carefully interpreted in light of the inherent differences among 
groups and those individuals who comprise the group. Group comparisons 
can be made validly when the groups as a whole are similar on certain basic 
characteristics. This section will review important characteristics which 
are related to achievement and which differ among the regions of the nation. 
These characteristics should be considered by all who seek to make appropri- 
ate comparisons. The nature of the national norm group will also be described 
in order that^state and national comparisons can be viewed from proper 
perspective. 

Characteristics of the National Norm Group 

In interpreting norm-referenced test scores in North Carolina, it is 

important for the reader to know the makeup of the norm group since the 

staters performance comparisons are against the standards set by the students 

who comprise this norm group. 

Prescribed conditions were set forth in the selection of students for 

inclusion in the ITBS norm group. The pupils in the sample had to be 

representative of the various ability and achievement levels in the nation. 

The sample had to be sufficient in number to represent adequately the many 

diverse elements of population within the United States. Accordingly, a 

sample size of slightly more than 20,000 students per grade was selected./ 

Sampling proWdures employed seven community size categories. A number 

to represent socioeconomic status was developed to further stratify each 
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size community. It was found that two important variables that measure 
socioeconomic status - (1) median years of education of the population 
25 years and over and (2) median family income, in thousands of dollars - 
had similar variability. Therefore, these two variables were summed to 
form an index of socioeconomic status which served as a basis for 
stratification of communities within a size category. 

In ordar to compensate for incomplete administrations due to absences 
and other losses, weights were assigned to all, raw score frequencies. Size 
of system, socioeconomic status, region, and public-parochial school balance 
were all considered in , assigning these weights. The weighted distributions 
are believed to closely approximate those of the total population of students 
in the United States for each grade. 

The following tables display the results of this sampling procedure. 

The reader can compare the national norm sample of pupils with the general 

population figures for both the nation and North Carolina. Table 2 shows 

a comparison of the percentages of students in the weighted norm sample 

that are within each of the national regions. The corresponding population 

» 

figures are shown for North Carolina and the United States. ^ ^ , 

TABLE- 2 

REGIONAL PERCENTAGES IN THE NORM SAMPLE AND 

THE TOTAL POPULATIONS OF N. C. AND U. 'S. 







-R 


E G I 0 N S 






GROUP. 


Northeast 


Midwest 


Southeast 


Southwest 


Far West 


Norm Sample 


21.4 


32.3 


19.9 


11.2 


15.2 


United States 
Population 


26.7 


29.1 


18.8 


ib.o 


15.4 


North Carolina 
Population 






■ 100.0 







Table 3 compares the percentage distribution of the students in the norm 
sample among seven conminity sizes with the percentages within these conwunity 
sizes in the nation and North Carolina. 

TABLE 3 

COMMUNITY SIZE PERCENTAGES IN THE NORM SAMPLE AND 
THE TOTAL POPULATIONS OF N. C. AND THE U. S. 





1 




1 


1 1 


1 




Lower B 




Communis 


iv Size 


Group 


1,000,000 


250,000 


100,000 


50,000 


25,000 


10,000 


1-10,000 


Norm Sanple 


9.7 


11.5 


7.7 


9.1 


8.5 


10.5 


A2.6 


United States 
















Population 


9.2 


11.5 


6.7 


8.3 


9.0 


10.2 


45.6 


North Carolina 


















0.0 


8.1 


23.2 


1.8 


8.2 


6.4 


52.4 



The socioeconomic categories within community size are not available 
from the test manual St however, the socioeconomic index ranged from 7 to 27. 
This range can aid in the interpretation of Table 4 which compares the U. S. 
and North Carolina populations. 



TABLE 4 

MEDIAN YEARS OF EDUCATION AND MEDIAN FAMILY INCOME (IN THOUSANDS OF DOLURS) I 
FOR THE 1970 TOTAL POPULATIONS OF N. C. AND THE U. S. 



Group 


Median Education 


Median Family Income 


Socioeconomic Index 


U. S. Population 
N. C. Population 


12.1 
10.6 


9.87 
7.77 


21.97 
18.37 ^ 
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Although ethnic membership was not a factor in selecting or weighting 
of the norm group, estimates of the percentages in the norm sample as well 
as the total North Carolina and the United States populations is given in 
the next table. 

TABLE 5 

ESTIMATES OF THE PERCENTAGE DISTRIBUTION WITHIN ETHNIC GROUPS 
IN THE NORM SAW>LE, THE NORTH CAROLINA SAMPLE, 
AND THE U. S. STUDENT POPULATION 















Group 


American- Indian 


Black 


Oriental 


Spanish/American 


White 


Norm Sample 


0.6 


15.7 


0.5 


4.0 


79.1 


United States 












Popul ation 










80.0 


(1968) 


0.4 


14.5 


0.5 


4.6 


North Carolina 












Sampl e 


1.2 


29.0 






69.0 



These tables indicate that the norm sample is representative of the 
nation and there are significant demographic differences be'c.-een pupils 
in North Carolina and pupils in the nation. 

Additional information about third graders in North Carolina and their 
environment is presented in the following tables. 
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TABLE 6 



PUPIL CHARACTERISTICS OF THE THIRD-GRADE ASSESSMENT SAMPLE 





Characteristic ' 


State Assessment Sample 


Sex 




. Male 


51. 3X 


reiwie 


AO CV 


D^/*4^1 /C^krk4/« llAinKAM«»U4r> 

KdCid i/t tnnic ncfnDersmp 




. American Indian 


1.2X 


. Black 


29.0X 


. White 


69. OX 


Parental Education Level 




. Neither over eighth 


5.9X 


. One over eighth 


25. 9X 


. One high school graduate 


44. 8X 


* One over high school 


23.5% 


Family Income Estimate 




. Less than $3»000 


15. 4X 


. $3»000 - $15»000 


75.6X 


. Over $15^000 


8.U 


Any Kindergarten Experience 




. Yes 


39.2% 


. No 


53.4% 


• Unknown 


6.8X 



North Carolina Comparisons With Other States 



Because learning does not stop at the end of the school day» it Is 
helpful to review the environment In which this outrof -school learning 
occurs. North Carol ina» the twelfth most populouSs state In the nation has 
been described as a "state of magnificent variety." The agriculture and 
Industry of North Carolina are varied > with the state producing two-thirds 
of the country's flue-cured tobacco and leader in fabric and furniture 



manufacture. Tourism also flourishes in North Carolina as thousands annually 
visit its mountains, parks, golf courses, shores, and sites for boating and 
fishing.' Clearly, such diversification makes it difficult to describe the 
"typical" North Carolinian. 

The diversity continues when such variables as individual income, 
occupatiorit race> and education are considered. These factors and the values 
placed on them vary not only by county and region^ but within conmunities 
as well. 

In studying educational status and change over time it is essential to 
examine achievement in conjunction with environmental factors. Of these 
elements, socioeconomic factors, in particular, are associated with 
educational opportunity and attainment, and these exert a major influence 
on a child's growth and development. Educators must consider theses tangibles 
and, more importantly, the values and ethics implicit in a child's immediate 
environment. The remainder of this section will contrast the environment 
of the North Carolina pupil with the environment of pupils in other states. 

A state's population, size, and- population distribution are basic 
environmental factors. The following tables show North Carolina's relative 
ranking. .More detailed information is included iij the Appendix. 



TABLE 7 

- NORTH CAROLINA'S RANK AMONG THE FIFTY STATES 
ON BASIC DEMOGRAPHIC FACTORS 



Factor 


North Carolina's Rank 


Land Area (1970) 

Population (1973) 

People Per Square Mile (1970) 

Percentage Classified Rural (1970) 

Percentage Black (1970) 

Median Age (1970) 


29 
12 
17 
5 
6 
15 
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As evident here, Nor'th Carolina students are from a more populous 
state where th^ peop'e are younge'^, more likely to be of a minority group, 
and live in smaller towns than people in most states. It is also apparent 
that the degree o^ rura^lty is twice that of the national average while 
the density is slightly above average. 



TABLE 8 

NORTH CAROLINA'S RANK AMONG THE FIFTY STATES 
/ON BASIC SOCIOECONOMIC FACTORS 







Factor 


North Carolina's Rank 


Per Capita Income (1972) 


34 


Households With Cash Incomes of 
$3,000 or less (1972) 


12 


Per Family Income (1970) 


40 


Median Years of Education (1970) 


46 



North Carolina's students do not share in as much of the basic 
socioeconomic wealth as do stUvlents from other states. That is, North 
Carolina ranks among the lowest ten states on important c^aracten sties 
such as 'income and education ieve! of adults- 

Thus, North CaroVna's combination of factors associated with income, 
ethnic composition, degree of ru^ality, and adult education level seem to 
indicate a "non-typical" backgrogind^^fpf her youth- A picture emerges of 
an environment which my not reinforce maximum educational progress. 

Comparisons Within North Carolina 
Just as there are strong differences between North Carolina and the 
average conpasite for the nation, there are great variations within the 
state's boundaries. Part'^cularly important is the variety which exists 



with the d1ffering*traditions ard personalities of its Mountain, Piedmont, 
and Coastal Plains groups. The following table describes some of these 
differences: «^ 

TABlE 9 

GENERAL ENVIRONMENTAL FACTORS WITHIN NORTH CAROLINA 



Factor 


Mountains 


Piedmont 


Coastal 
Plains 


^ State 


Population 
(1970) 


760,760 
(15%) 


2,69^,975 
(54%) 


1 .628,323 
(32%) 


5 ,C82 .059 


Growth 
(1960-1970) 




2K3% 


7.7% 


11.5% 


Distribution of 
Black Population 
(1970) 


41,459 . 
(4%) 


569 .575 
(51%) 

/ 


515.444 . 
(46%) 


1,126,478 


Percen^tage Black 
(1970) 




. 5.4%. 


/ 
/ 

21.1% 


31.6% 


22.2% 


Percentage Classified 
Rural (1970) 


75.1% 


45.9% 


60.6% 


55.0% 


Percentage That Moved 
(1965-70) 


•40.5% 


1 46.0% 


49:2% 

^-^ - 


46.2% 



These basic environmental factors indicate that a majority of th/ peopt 
black and white - live in the Piedmont; the Mountains have* the highesl J 
percentage of the people living in rural areas;, and the Coastal Plains 
popUlatTon has a greater proportion that is black. Perhaps the-major point 
in these figures is the variety between these three major geographical 
diyisicns. As disclosed earlier some of these same variables have been 
shown to be related no achievement. 
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Distribution of economic resources in these three regions also varies 
as(the following table shows: 



TABLE 10 



SOCIOECONOMIC FACTORS WITHIN NORTH CAROLINA 













f 

Factor 


Mountains 


Piedmont 


Coastal 
Plains 


State 


Family Income 


8,059 


10,234 


7,757 


9,139 


Fami ly Income 
Female Head (1970) 


5,017 


5,620 


4,104 


5,017 


Average Percentage 
Fr*ee School Lunch 


35.2% 


37.6% 


64.7% 


47.8% 


pprrpntaop Li Vina 
Below Poverty 


20,2% 


15. n 


28.8% 


20.3% 


Percentage of all 
Families Below Poverty 
with Children Under 18 


10.7% 


8.9% 


19.2% ' 


12.3% 


Percentage of all Children 
Under 18 From Poverty 
Families 


20.5% 


17.4% 


34.4% 


^3.6% 

/ 


Percentage of Children 
Under 18 Living with 
Both Parents- 


82,6% 


80.1%^ 

r 


73.7% 


78.3% 



Because socioeconomic status is' a strong predictor of academic sWcess, 
regional differences In educational achievement are to be expec]:ed^ Thus, 
any academic cojnparisons should be carefully tempered by these background 
differences. 
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still another factor associated with academic achievement is the 
educational environment. Regional patterns are suggested in the table below: 

TABLE 11 

EDUCATIONAL FACTORS WITHIN NORTH CAROLINA/ 













Far tor 


Hounta i ns- 


P iedmont 


Coastal 
Plains 


State 


Average of Median Years 
of Education - Adults 
Over 25 


9.5 


10.2 


9.9 


10;6 


Adult Education Index 


2.50 


2.82 


- 2.56 


2.69^ 


Percentage of High School 
Graduates of Those 16-21 
Not In School 


49.7% 


48.7% 


44.0% 


46.7% 


Taxing for Education 
index 




507 


439 


478 



These environmental, socioeconomic, and educational factors are a major 
influence on a child's educational growth and development. Educators who 
consider regional comparisons must be aware of the differential effects 
that these factors contribute within regions. Certainly, expectations are 
better determined with an awareness of the status of these variables 
irrespective of whether local, regional, or state comparisons are being 
made. * — 1 



HIGHLIGHTS 



Two tests were used to measure mathematics achievement of third graders 
In the 1974 State Assessment of Educational Progress: the Iowa Tests 
of Basic Skills and the objective-based SCORE Mathematics Test; Major 
findings were as follows: 

. North Carolina's third grade students, on the average, scored several 
months below the nation in mathematics on the standardized Iowa Tests 
of Basic Skills. 

. Approximately 40 percent of both North Carolina's and the nation's 
students scored within the average4)r third grade score interval. 
However, more of North Carolina's students scored below the average 
or third grade score interval than did the nation's students, and 
more of the nation's students scored above the average interval than 
did North Carolina's third graders. 

. The ranges of the national and North Carolina score distribution are 
quite similar, but proportionately more of North Carolina's students 
than the nation's students scored in the score interval just below 
the average score interval. 

. This difference in the score distributions is reflected in the math- 
ematics grade equivalent averages for North Carolina (31.1) and for 
the nation (38.0). 

. North Carolina and the nation differed little on the percentage of 
students correctly responding to individual items of the ITBS. gen- 
erally, the national item percentage achievement exceeded North 
Carolina's achievement by six to eight percentage points. 

. North Carolina's third grade students scored satisfactorily on a 
iwjority of the SCORE mathematics objectives. ^ 

. North Carolina's students scored well in the broad areas of measure- 
ment, counting, basic computations, geometry and graphs. 

. North Carolina's students scored somewhat below standard on the broad 
areas concerning the U. S. tnonetary system, modern mathematics con- 
concepts, complex computational problems, and fractions. 
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MATHEMATICS ACHIEVEMENT 

As part of the 1974 State Assessment of Educational Progress, the 
mathematics achievement of 5,000 randomly selected third graders was 
measured in April, 1974. Approximately half of the 5,000 students were 
given the Concepts and Problem Solving subtests of the Iowa Tests of 
Basic Skills (ITBS). The other half of the students were tested^with 
the objective-referenced Mathematics Test developed by the State Depart- 
ment of Public Instruction's Division of Mathematics in conjunction with 
SCORE (School Curriculum Objective-Referenced Evaluation), a service for 
creating customized objective-referenced tests. 

This report will describe the two complementary tests used in the 
mathematics assessment and will report the students' performance on both 
of them. It will also present a summary of teachers' and principals' 
perceptions of their mathematics programs. 

Summary of Principals' and Teachers' Perceptions 
of Their Mathematics Programs 

' Before considering the descriptions and results of the ITBS and the 
SCORE Mathematics Test, it might be helpful to summarize principals' and 
teachers' views concerning their mathematics programs. This information 
was obtained through two surveys which were part of the 1974 State 
Assessment of Educational Progress: the "School Information Question- 
naire" for principals and the "Teacher Questionnaire." There were 618 
principals and 12,875 K - 6 teachers ^and other educators responding to 
the random-sample survey. 
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Over 86 percent of the elementary teachers report that their pro- 
fessional training in mathematics teaching was at least adequate, and 
90 percent of the teachers state that they are proficient in the teach- 
ing of mathematics. Despite these positive perceptions, teachers (as 
well as principals) recognize several arieas which they believe nega- 
tively affect mathematics instruction. 

Specifically, teachers and principals agree that a lack of supplies 
and materials is a serious handicap to effective mathematics instruction. 
In fact, teachers ranked mathematics as the second academic subject, 
behind science, that is most affected by this obstacle. In addition to 
this problem, principals indicate a need for specially trained teachers 
to assist the classroom teachers in the area of mathematics. Mathemat- 
ics was judged as the second academic subject, after reading, which 
would most benefit by the addition of specially trained personnel. 

Only one-third of the responding elementary teachers report using 
the State Department source book, "Mathematics Goals and Activities," 
and only two out of five teachers indicate that they have attended a 
mathematics workshop within the last three years. 

For further information concerning the principal and teacher ques- 
tionnaires, the reader is referred to the Division of Research report, 
"Educational Progress in Grade Three: A Survey of Teachers and 
Principals." 
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Description of ITBS and SCORE Mathematics Tests 
I TBS Description 

The mathematics skills items of the ITBS are grouped into two sub- 
tests (Concepts and Problem Solving) paralleling the textbook offerings 
throughout the nation. The ITBS provides two subtest scores as well as 
a total mathematics achievement score. 

Both subtests cover most of the following areas: U. S. monetary 
system, equations, fractions, geometry, measurement, equalities, in- 
equalities, numeration, and whole numbers, and numerical computation. 
Although both the Concepts and Problem Solving subtests deal with the 
same areas, they approach the testing of these areas from opposite view- 
points. In the Concepts section, the emphasis is on the understanding 
of our number system— its terms, processes, and operations. Therefore, 
this subtest deals with the logic of the computational process. In con- 
trast, the Problem Solving section tests numerical computation. Thus, 
mathematics c/)mpetence is tested in both a theoretical and applied 
setting. 

I 

SCORE Test Description 

For diagnostic purposes, the Division of Mathematics of the State 
Department of Public Instruction supplemented and expanded the concepts 
measured in the ITBS by developing, the objective-referenced Mathematics 
Test, with the assistance of SCORE. This test was constructed by se- 
lecting test items to measure specific math objectives. Specifically, 
the SCORE Mathematics Test expanded the ITBS areas of measurement, 
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aquations, inequalities, geometry, whole numbers ^. and numerical compu- 
tation and introduced items dealing with sets, graphs, and number se- 
quences. For a listing of the SCORE mathematics objectives the reader 
is referred to the Appendix. ' 

Thus, through the use of both the ITBS and SCORE mathematics tests. 
North Carolina's third grade students Were exposed to most, if not all, 
of the mathematics objectives thought to be important at this develop- 
mental level. 

- -J' 

ITBS Mathematics Results 

Comparison of State and National Averages 

To explain the results, it is necessary to compare the distribution 
of North Carolina's student scores with that of the national sample. 
These score distributions are presented in Table 1. 

TABLE 1 

' PERCENT OF NORTH CAROLINA AND NATIONAL STUDENTS 

SCORING AT VARIOUS GRADE EQUIVALENT RANGES 
ON THE ITBS MATHEMATICS TEST 



Area 


Grade Equivalent Ranges 


00-19 


20-29 


30-39 


40-49 


50-69 


Nation 
N. C. 


2.0% 
9.4% 


22.0% 
33.4% 


39.0% 
37.8% 


28.0% 
16.5% 


9.0% 
2.9% 



Table 1 reveals that approximately 40 percent of both North Caro- 
lina's and the nation's sample scored within the average or third grade 
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interval (grade equivalent 30-39). However, more of North Carolina's 
students scored below the average on third grade interval than did the 
nation's students, and more of the nation's students scored above the 
third grade interval than did North Carolina's third graders. 

Jhus, both similarities and contrasts are detected in the basic 
pattern of the North Carolina student score distribution and the nation- 
al student score distribution. The ranges of both score distributions 
are quite similar, but the two distributions are different in that pro- 
portionally more of North Carolina's sample than the national sample 
scored in the score interval just below the average score interval. 

This difference in the score distributions is illustrated by 
comparing the graph of North Carolina's distribution with the graph of 
the national distribution. These graphs are presented in Appendix A. 

This difference in the score distributions is naturally reflected 
in the grade, equivalent averages which were computed for North Caro- 
Tina's and the nation's mathematics achievement scores. The grade 
equivalent average is 38.0 for the national sample and 31.1 for the 
North Carolina sample. Thus, if it is assumed that educational develop- 
ment is constant throughout the year, there is a seven month developmen- 
tal difference between North Carolina and the nation at the third grade. 
However, this explanation is not completely tenable because educational 
development does not occur uniformly throughout the year. Therefore, 
perhaps a better interpretation of the results would be; (1) North 
Carolina's third grade students did, on the average, score several 
months lower on this test than did the nation's third grade students; 
(2) fifty-seven percent of North Carolina's third grade students, who 

/ 
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were given the test in April , scored within or above the third grade 
equivalent interval; (3) seventy-six percent of the pupils in the nation 
scored within or above the third grade equivalent interval. 

ITBS Subgroup Results 

In addition to the state and national comparisons reported above> 
the scores computed for third grade pupils within North Caroling were 
also analyzed In terms of region, race/sex category, family Income 
level, and parental educational level. Since the averages for both 
subtests followed the same general score patterns as did the /composite ' 
average for each of the classification variables, only the composite 
average will be reported. For subtest Information the reader Is re- 
ferred to Appendix A. . ^ 
ITBS Results by Region 

- > / 

On the average. Piedmont students attained about the same compos- 
ite score as the state. Itountain third graders averaged slightly above 
state, while Coastal Plains students averaged somewhajt below the state. 

In specific terms. Coastal Plains third graders, on the average, 
scored 29.9 (one month below the state average). Piedmont students 
scored 31.2, which is approximately the state average, and the Mountain 
students scored 34.0 (three months above the state average). 
ITBS Results by Race and Sex / 

When North Carolina student scores were analyzed by the combined 
variable of sex and race, it was found that little difference existed 
between averages for black males and black females and between averages 
for white males and white females. The averages fov these four groups 
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are 25. 0» 25.3, 33.6, and 33.6, respectively. Thus, the averages for 
black males and black females are approximately six months below the 
state average, while the state average is approximately two to three 
months below the average for white miles and white females. 
ITBS Results by Parental Education Level 

Students having at least one parent who graduated from high school 
averaged 32.3 on the composite ITBS mathematics score (one month above 
the state average). Scoring above the state^. average by almost six 
months were the students who had at least one parent with post<*high 
school training (36.6 average). Students having both parents with less 
than an eighth grade education averaged 26.0 (five months below the 
state average). The average was 27.8 (three months below the state 
average) for those students having at least one parent who completed 
more than eighth grade but did not complete high school. 
ITBS Results by Family Income Level 

Students whose family income is above $15,000 averaged 38.6 (eight 
months above the state average). Those whose family income is below 
$3,000 averaged 25.6 (six months below the state average). 

Summary of ITBS Subgroup Results 

From these analyses, it is apparent that the average scores differ 
for various subgroups of North Carolina's third grade students. However, 
it should be remembered that these are just averages. Within any given 
classification there are students who score just as high or just as low 
as students in other classifications. The difference lies in the fact 
that a greater proportion of students within each classification scored 
either above or below the state average. 
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ITBS Diagnostic Results 

In addition to the previous analysis, item difficulty analysis was 
oerformed on each subtest of the ITBS mathematics test. It was found 
that North Carolina's students responded in much the same way with re- 
spect "to the difficulty of each item as did the nation's third graders. 
However, for any given item the percent of correct responses differed 
for North Carolina and the nation (see Appendix B). On some items, 
a greater percentage of the nation's students answered correctly, while 
on other Items a greater percentage of North Carolina's students an- 
swered correctly. Specifically, on the Concepts subtest. North Caro- 
lina's students ranged from three percentage points higher than the 
nation on some items to 17 points below on others. The best estimate is 
that about eight percent more of the nation's third graders correctly 
answered the itemsf found in the Concepts subtest. 

On the Problem Solving subtest,. North Carolina's item difficulty 
plot closely follows that of the nation. The percent of North Carolina 
students responding correctly to each item ranges from three percentage 
points above the rwtion to 14 points below. This information might be 
interpreted to say that on the average, approximately six percent more of 
the nation's third graders answered these subtest items correctly. 

Using as a standard the percent of national third graders correctly 
responding to an item, it may be concluded that North Carolina's students 
scored best fn the areas concerning the reading of time, computing units 
of measurement, operating with number sentences and sequences, recog- 
nizing numbers and subsets of numbers, and computing simple addition, 
subtraction and multiplication problems which require no regrouping 
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operations. On the other hand, there were several areas on which North 
Carolina students scored noticeably below the nation. These areas 
covered problems involving U. S. currency and weight, applying the 
number line, understanding mathematical properties and laws of the 
number system, estimating the value of w.hole numbers, and rounding off 
whole numbers. 

SCORE Test Results 

> 

The objective-referenced SCORE Mathematics Test vwis designed to 
expand and Supplement the ITBS Math Test. 'However, unlike ITB'S items, 
SCORE items were devised to measure student mastery of specific majthe- 
matics objectives. Analysis of an objective-based test such as the SCORE 
Mathematics Test requires compdrison of student performance to a stand- 
ard rather than comparison of an individual's or group's performance to 
the performance of a norm group. 

In assessing the qualuy of the objective-based SCORE test, the 
following questions should be asked: *'Have the objectives been judged 
appropriate and/or important for North Carolina's third grade students?" 
"How well did the items measure the objectives?" "How difficult were 
the items for third grade students?" 

To provide answers to these questions, outstanding third grade 
teachers were selected as respondents for two questionnaires concerning 
objective and item appropriateness and item difficulty. (See Appen- 
dix C.) Since it is necessary to refer to teachers' responses while 
discussing the student achievement on the SCORE test, students' results 
will be categorized and reported by five general areas specifically 
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developed to siff.marize the teachers' questionnaire data. These five 
"objective clusters" will be presented in descending order of objective 
importance. Because teachers overwhelmingly rated all the test items as 
appropriate measures, the factor of item appropriateness is deleted from 
the categories below. 

Objective Cluster One 

In the first cluster, which dealt with such basic skills areas as 
recognition of numerals and performing basic calculations, the objec- 
tives were judged as being very important for and generally being taught 
to North Carolina's third grade students. In addition, the items de- 
signed to measure these objectives were judged as not being very diffi- 
cult for third graders. 

On the average, between 84 percent and 90 percent of North Caro- 
lina's students correctly responded to items measuring this group of 
objectives. In view of the questionnaire information, the conclusion • 
was drawn that North c^^rniina's students demonstrated a high level of 
performance on these objectives. (See Appendix C.) 

Objective Cluster Two 

The second group of objectives dealt with such mathematics areas as 
measurement techniques and number properties. These objectives were 
also judged very important for and now being taught to North Carolina's 
third graders, but the items measuring these objectives were rated as 
somewhat more difficult than items in the previous category. 
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On the Cluster two objectives, North Carolina's students scored 
somewhat lower than;pn the Cluster On^ objectives. However, they demon- 
strated satisfacto^^achievement, because the percent of students cor- 
rectly responding to these items ranged from 62 percent to 83 percent. 
Two exceptions to this statement should be noted. Sn the areas requir- 
ing the stud.ent to recognize and apply inequality symbols in number 
statements ancj to recognize a number and its equivalent number statement. 
North Carolina's students scored somewhat below standard. 

* ^ Objective Cluster Three 

jr 

A third category of objectives tested the students* knowledge in 
the broad area of geometry and in .the interpretation of graphs.. These 
objectives were judged to be less important than, objectives in Clusters 
One and Two and, in general, were reported as not being taught at the 
third grade. However, the itens associa^ted with these objectives were 

c 

not deemed too difficult. 

Between 87 percent and 96 percent of North Carolina's students chose- 
the appropriate response for this group of items. (See Appendix C.) 
Thus, there was a high degree of achieve[nejit.jon these objectives even 
though they were not^.5aneratTy being taught in North Carolina's third 
grade mathematics programs. 

Objective Cluster Four 

\ ' . 

The fourth objective cluster, which included such areas as' sub- 
traction ard the checking of subtraction problems by addition, was 
evaluated as being generally important. Further, these objectives were 

33 



reported as generally being taught at the third grade level. The items 
designed to measure these objectives were judged to be fairly difficult. 

On these objectives North Carolina's third grade students demon- 
strated a less than desirable achievement leveiVwith between 34 percent 
and 65 percent of tte students responding correctly. 

Objective Cluster Five \ 

Finally, one obipctive which dealt with fractions of sevenths, 
eighths, and tenths was judged as inappropriate for third grade studentis. 
In addition, this objective was reported as generally not being taught 
to North Carolina's third graders, and the items for this objectiv were 
judged as very difficult. 

On this objective only 38 percent of the students responded cor- 
rectly. It is wrong to conclude from this that North Carolina's stu- 
dents scored unsatisfactorily, because this objective was rated as 
inappropriate. However, it should be reported that student performance 
on other objectives dealing with simpler fractions was judged to be 
slightly below standard. 

Summary of SCORE Results 

In conclusion. North Carolina third grade students scored at least 
satisfactory /ton a majority of the SCORE mathematics objectives, while 
performing less than satisfactorily on others. 

It should be remembered that the informaiclon just presented has 
been a summary. For a more detailed presentation of the SCORE Mathe- 
matics Test, the reader is referred to Appendix D which presents the 

34 



ERIC 



results of the SCORE item cluster analysis. 

Conclusion 

The final question remains: "What do all these analyses say about 
the basic mathematics skills of North Carolina's third gradi students?" 

Some answers are as follows: (1) North Carolina's third grade 
students, on the average, scored somewhat below the nation on the 
standardized ITBS Mathematics Subtests, (2) North Carolina's third 
grade students scored satisfactorily on a majority of the SCORE mathe- 
matics objectives, (3) North- Carolina's third grade students ^id well 
in the broad areas of measurement, counti».ij, basic computations, 
geometry and graphs, (4) North Carolina's third grade students scored 
somewhat below standard on the broad areas concerning' the U."S. monetary 
system, modern mathematics concepts, complex computational problems, and 
fractions. 
I 
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MATH TOTAL AND SUBTEST AVERAGE GRADE EQUIVALENT 
SCORES FOR NORTH CAROLINA AND VARIOUS CLASSIFICATION VARIABLES 





Subtests 


Math 




Concepts 


Problem Solving 


Total 


cTA ire 
5TATt 


30.7 


31.4 


ol • 1 


CLASSIFICATION VARIABLES 








toasxo 1 r lams 

Piedmont 

Mountain 


29.9 
30.6 
33. F 


29.9 
31.6 
34.2 


31.2 
34.0 


Male 
Female 


30"7 
30.8 


31.4 
31.3 

• 


31.2 
31.0 


Black 
White 


24.6 
33.3 


25.6 
33.8 


25.1 
33.6 


PARENTAL EDUCATION LEVEL 
Neither over eighth 

. 1 high school graduate 
1 over high school 


25.2 
27.1 
32.0 
36.7 


26.2 
28.3 
32.4 
36.5 


26.0 
27 8 
32.3 
36.6 


FAMILY INCOME 
Low (under) $3»000 
Medium ($3.000-$15.000) 
High (over $15,000) 


25.1 
31.1 
3^.3 


26.0 
31.7 
38.8 


25.6 
31.5 
38.6 
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TEACHERS' RATING OF MATHEMATICS ASSESSMENT \)BJECTIVES 
FOR SCORE MATH TEST 

Instructions 

The third-grade Mathematics Test in the 1973-74 State Assessment of Educa- 
tional Progress included objectives not eidequately measured by the standard- 
ized test that was given. Twenty intermediate or more general objectives 
were identified, and one or more instructional or student-performance objec- 
tives were selected for each intermediate objective. Each intermediate 
objective is identified by a Roman numeral; its instructional objective(s) 
is (ar*e) numbered consecutively after it. - 

You are requested to rate each of the objectives (both intermediate and in- 
structional) for three areas of concern: Importance, Being Taught, District 
Curriculum. There are three rating columns, one for each area. Circle one 
number in each column which best reflects your thinking. 

Column A, Importance In this column you should rate the objec- 

tive for its overall importance to third- 
grade students, Is it important for these 
students to know or to be able to do? Does 
it reflect an important learning skill for 
them? You should judge here on the general 
importance of the objective and not whether 
it Is currently being taught. Circle one 
number. 

1. Very Important 

2. Somewhat Important 

3. Not Very Important 

Column B, Being Taught: Here you are asked to rate the objective as 

to whether It Is being taught to your students. 
That is, do you teach to this objective? If 
you are a member of a team teaching situation 
andNio not teach this particular subject, 
pleasti make a judgment as to whether other 
team t^chers teach to tfiTs^bjective. (This 
will help us to know if the objectives 
selected are ones that the students in North 
Carolina generally encounter.) Circle one 
number ^ 



ERiC 



1. YeSt I (or team member) teach to 
this objective. 

2. ^This objective is not specifically 
covered with my students* but they 
have highly similar content and 
activities. 

3. No» my students are not taught 
this objective. 

Column C, District Curriculum: This rating asks for your best judgment as 

to whether this objective is generally 
covered in the curriculum throughout your 
local school system. Thus, you might 
teach an objective which few others teach 
and would mark "Ho." Or you might not teach 
a particular objective which many others 
might teach and would mark "Yes." This 
' rating is, of course, very subjective, and 

the results will be considered only as 
general indicators. Circle one number. 

1. Yes, this objective is generally 
taught. 

2. No, this objective is not 
generally taught. 

3. I do not know. 



The numbers in the rating columns have headings only on the first page of 
the questionnaire. In order to keep in mind the meaning of each number, 
refer back to the instructions (which may be separated from the question- 
naire if desired) or the front page of the questionnaire. 
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Column B, Difficulty: 



TEACHERS RATING OF OBJECTIVE- ITEM 
CONGRUENCE AND ITEM DIFFICULTY 
FOR SCORE MATH TEST 

Instructions 

In this questionriaire you are asked to look at each item on the Math Test 
and make a judgment about how' well it seems to measure the objective for 
which it is listed. Two types of ratings are requested: one dealing with 
how well you think the item measures the objective with which it is listed, 
and the second dealing with how difficult you think the item would be for 
your students. 

Column A, Measure: Here you are to judge the excellence of the item 

as a measure of the objective listed. Circle 
the number that best reflects your own opinion. 

1. A very good measure of the stated objective 

2. An adequate or generally acceptable measure 

of the objective 

3. A poor measure of the stated objective 

Please rate each item on its difficulty level 
or on what percentage of third-grade students 
you have taught which you think would pass that 
item in the spring. Ratings range from "1" 
(Very Difficult) to "5" (Very Easy). In order 
for the responses from all the raters to be 
comparable, please. use the following guidelines 
for ranking each item. The percentages represent 
the number of your students you believe could 
answer the item correctly. 

1. 0-20% answer correctly 

2. 21-40% answer correctly 

3. 41- 60% answer correctly 

4. 61-80% answer correctly 

5. 81-100% answer correctly 

Below each objective, the item(s) measuring that objective is (are) listed. 
You must refer to that item number in the enclosed Math Test booklet. In 
order to completely understand some of the math items, the instructions for 
the test administrator are also necessary. Therefore, we have combined the 
administrator instructions with the test items so you only have to refer to 
one booklet. For example, items one and three are measures of the first 
objective listed. You should turn first to item number one in the test 
booklet and read the item completely. After completing the item, rate it 
on the two scales described above: Measure and Difficulty. Then do the 
same for item number three. Follow this procedure for the items listed 
for each objective; 
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The test items are generally boxed in to separate them from any administra 
tor's instructions. When reviewing the items, keep in mind that all items 
were read aloud to the students in order to reduce the influence of poor 
reading on the ability to answer. 

You may refer back to these instructions if necessary in order to keep 
the rating scales in mind. If helpful, you may separate this sheet of 
instructions from the rating questionnaire itself. \ 
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ITEM RATING SCALE 









1- Good 

2- Adequate 

3- Poor 


1 : 0-20% 
2: 21-40% 
3: 41-60% 
4: 61-80% 
5: 81-100% 




Objectives 


Item 


A 

Measure 


B 

Difficulty 




The student will be able to 
identify sets, set membership. 
















The student will be able to 
identify in a pictorial repre- 
sentation which includes both 
objects in a set and objects 
that are not in the set, those 
objects that are members of the 
set. 


1 
3 


76 
1 

81 


2 
2 

19 


3 

3 


1 2 3 4 5 

7 21 36 37 

1 2 3 4 5 

7 Ik 33 i»5 


IL 


The student will demonstrate 
an unaerstanaing ot the concept 
of number. 
















The student will be able to 
identify the position in space 
or time of a particular object , 
by naming the corresponding \^ 
ordinal number. 


6 

T ft 

19 


1 

6t» 
1 

86 


2 

29 

2 

Ik 


3 

7 

3 


1 2 3 4 5 

7 12 2k 29 29 

1 2 3 4 5 

2 2 19 36 kl 


III. 


The student will be able to 

read, write, and identify numerals. 
















The student will be able to write 
the number given orally. 


50 


1 

86 


2 

10 


3 

5 


1 

5 


2 3 4 5 

5 5 10 76 




The student will be able to write 
a fraction for the rational number 
associated with a model of halves, 
thirds or fourths. 


23 


1 

67 


2 

26 


3 

7 


1 2 3 4 5 

7 26 31 31 5 




The student will be able to write 
a fraction for the rational number 
associated with a model of 
sevenths, eighths, or tenths. 


18 
22 


1 

76 
1 

71 


2 

19 

2 

27 


3 

5 

3 

2 


1 

12 
1 

17 


2 3 4 5 

17 32 17 12 

2 3 4 5 

i»l 21 14 7 


IV. 


The student will demonstrate an 
understanding of regrouping and 
the use of expanded notation. 
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l-6ood 










2-Adequate 


2: 21-40« 






3-Poor 




3: 41-60« 










4: 51-80« 










5: 81-100% 






A 




B 


Objectives 


Item 


Measure 
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The student will be able to write 
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notation. 




V. The student will demonstrate an 










understanding of the decimal 










numeration system (place value) • 










The student will demonstrate a 
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1 2 3 
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2 3 4 5 


knowledge of place value by 




67 31 2 


12 


21 17 33 17 
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in four-digit numerals. 
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2 3 4 5 
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97 17 9U 

A/ 


VI- The student will demonstrate a 










knowledge that there are many 










names for the same number. 










The student will demonstrate a 
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knowledge that many different 
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vUinu 1 iia L 1 uiid ui iiuiiicia o auucu 






together can equal the same 










specific number by selecting 










a combination of numerals that 
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would be the same as a specified 




86 12 2 
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5 21 21 50 


number. 






The student will be able to write 
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1 2 3 


1 


2 3 4 5 


different numerals which when 
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added together or subtracted 
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indicated number. 
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VII. The student will demonstrate an 










understanding of the order 










property of whole numbers. 










Th*p ^tuHpnt will he able to Dut 
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groups of 1 to 10 objects in 
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order from t^e largest to the 






smallest ai >om the smallest 










to the largt-v. 











78 



Objectives 



Item 



1- Good 

2- Adequate 

3- Poor 



1: 


0-20% 


2: 


21-40% 


3: 


41-60% 


4: 


61-80% 


5: 


81-100% 



A 

Measure 



B 

Difficulty 



The student will be able to 
Identify a number as being 
greater than, equal to, or 
less than a second number* 

VIII. The student will demonstrate 
an understanding of the theory 
of multiples. 

The student will be able to 
count the elements of a non- 
empty set by twos, fives, 
tenSt etc., to determine the 
number of elements in the set. 

The student will demonstrate a 
knowledge of skip counting by 
skip counting by 3, 5, 10, 
100, 1000. 

The student will be able to 
supply missing multiples in an 
incomplete sequence of 
multiples^ 

IX. The student will demonstrate 
and understanding of 
properties of addition 
and subtraction. 

The student will be able to 
write the related subtraction 
equation for a two-number 
addition equation with a 
missing addend. 

X. The student will demonstrate 
an understanding of the 
properties of mil tiplication 
and division. 
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Objectives - 



The student will apply the 
knowledge that multiplication 
makes an answer have a larger 
number and whole number division 
makes an answer have a smaller 
number, by being able to select 
either the multiplication or 
division signs which would belong 
in a specified problem 

XI. The student will demonstrate an 
ability to add whole numbers with 
two or more digits. 

The student will be able to add 
two one-, two-, three-, or four- 
digit numerals with no regrouping 



' The student will be able to 
write the sum of an addition 
problem with two addends of 
three- or four-digit numbers. 

XII. The student will demonstrate 
an ability to solve subtraction 
problems with two or more digits, 

The student will be able to 
subtract a two- or three-digit 
number from a three-digit 
number with regrouping. 

The student will be able to find 
the difference of two one-, two-, 
three-, and four-digit numerals 
with no regrouping. 

The student will be able to ctieck 
a two-digit subtraction equation 
by adding the remainder to the 
subtrahend. 
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1- Good 

2- Adequate 

3- Poor 



0-20% 
21-40% 
41-60% 
61-80% 
81-100% 



A 
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Difficulty 
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7 12 33 52 
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36 55 
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Objectives 



Item 



l-Good 


1: 


0-20X 


2-Ad equate 


2: 


21-40X 


3-Poor 


3: 


41-60X 




4: 


61-80% 




5: 


81-lOOX 



A 

Measure 



B 

Difficulty 



XIII. The student will demonstrate an 
understanding of measurement 
involving the United States 
monetary system. 

The student will be able to 
indicate the value of a given 
group of pennies, nickels, and 
dimes that total less than $1.00 

XIV. The student will demonstrate an 
understanding of the measurement 
of time. 

The student will state the time 
shown on a clock face to the 
nearest minute. 

. XV. The student will demonstrate 
an understanding of the English 
System of measurement. 

The student will be able to 
measure an object to the 
nearest inch using a twelve- 
inch ruler marked with 
fractions of inches. 

XVI. The student will demonstrate 
an understanding of geometric 
figures and their properties. 

The student will draw a closed' 
curve with the point on, inside, 
and outside the curve. 

XVII. The student will demonstrate an 
ability to make geometric 
drawings and constructions. 
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Objectives 



Item 



l-Good 


1: 


0 


-20% 


2-Adequate ^ 


2: 


21 


-m 


3-Poor 


3: 


41 


'60% 




4: 


61 


-80% 




5: 


81 


-100% 



A 

Measure 



B 

Difficulty 



The ^ student will be able to 
make rough drawings of 
circles, squares, rectangles, 
triangles, and line segments, 

XVIII. The student will demonstrate 
an ability to compute the 
area of geometric ^figures, 

• The student will be able co 
state the number of square 
units contained in a region 
of a square ^nit grid. 



XIX. The student will demonstrate 
■ ^an ability to interpret 
graphs and tables. 



The student will be able to 
-aivswer-'<}uestions related to 
a bar graph. 



16 
17 



8 

15 



9 
11 



XX. 



The student will , demonstrate 
an ability to analyze number 
sentences to determine their 
truth and appropriateness. 

The student will be able to 
indicate the proper relationship 
by supplying the correct symbol 
in a closed mathematical 
sentence with a missing order 
symbol • 
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SCORE MATH OBJECTIVE CLUSTERS 



Cluster One 

Operational Symbols 
Counting by Multiples 
Sets 

Reading and Writing Numerals 

Addition (Three- and Four-Digit Addends With No Regrouping) 
Measurement (Length) 



Measurement (U, S. Monetary System) 
Equalities 

Number/Numeral Distinction Involving One Operation 
Ordinals (Position) \ 
Number Sequence 

Subtraction (Three- and Four^Diglt Addends With No Regrouping) 

Cluster Three ^ 

Non-Metric GeomeVy 
Metric Geometry-' 
Graphs 

Cluster Four 
Measurement (Time) 

Addition (Three- and Four-Digit Addends With Regrouping) 

-5iit>trdet-i^n~{^hree — a nd Fou r -Digit A ddends With R e gr ouping)- — 

Subtraction Checked by Addition 

Properties of Fundamental Operations 

Place Value and Expanded Notation 

Inequalities 

Fractions (Halves, Thirds, Fourths) 

Number/Numeral Distinction Involving Two OtJera^M^ns""^ 

Ordinals (Order Property) 

Cluster Five 

Fractions (Sevenths, Eighths, Tenths) 
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EXPLANATION OF TABLE ENTITLED. "PERCENTAGE OF STUDENTS 
' CORRECTLY RESPONDING TO VARIOUS NUMBERS OF ITEMS 
WITHIN EACH OBJECTIVE ON MATH TEST" 

The following table presents the percentages of students correctly 
responding to various numbers of items measuring each of the 26 objec- 
tives on the SCORE Mathematics Test- Analyjsis of these percentages 
reveals approximately how well North Carolina's third grade students 
have achieved basic mathematics skills. Results are shown on the table 
according to the following classifications; state, region (Coastal 
Plains, Piedmont* Mountain), race by sex (black male, ^ black female, 
white male, white female), family income (high, middle, low), and paren- 
tal education level (1, 2, 3, 4). 

However, several cautions are needed for interpreting this informa- 
tion. (1) It is important to keep In mind the method used to construct 
an objective-based test. (2) It is crucial to know whether a given 
ODjecTTve~Ts"considered imporEafvE for North Carolina's students. (3)" 
The percentages reported should not be confused in any way with a grading 
scale;, i.e., a 70 percent figure does not imply a pass or fail point* 
(4) There is no constant mastery level used here for all objectives 
due to the arbitrary nature of such a mastery level, the differing number 
of items used to measure various objectives, and the heterogeneity of 
North Carolina's student population. 

To further clarify these concepts, two examples of objective achi'^ve 
ment data interpretation are presented. 

For Objective 1, 97 percent of North Carolina's students correctly 



i 



responded to at least one of the four items designed to measure this 
objective; 88 percent correctly responded to at least two; 67 percent to 
at least three, while 29 percent correctly answered all four'. Do these 
figures imply that North Carolina's students did or did not perform 
satisfactorily in relation to this objective? 

Since almost 90 percent of North Carolina's third grade sample 
correctly answered half of the items of Objective 1 and over two-thirds 
of the students correctly responded to three out of four of the items. 
North Carolina's students demonstrated a high degree of achievement on 
this objective. 

For Objective 6, 74 percent of the students answered at least one 
of two items correctly, but only 16 percent^ correctly answered both of 
the items. North Carolina's students performed less than satisfactorily 
on this objective. 

These two examples, plus the following achievement data summary 

-^ihld-^ri^^+^^nt^rpretairhm-of^th 

pages, 

STATE SUMMARY OF OBJECTIVE ACHIEVEMENT DATA 

I. Objectives judged to be satisfactorily achieved by North Carolina's 
third graders: 

1, 3, 4, 5, 9, 10, 11, 12, 14, 15. 18, 19, 20, 22, 24 

II. Objectives judged to be unsatisfactorily achieved by North Carolina's 
third graders: 

2, 6. 7, 8, 13, 16, 17, 21, 23, 25, 26 
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NORTH CAROLINA AND NATIONAL DEMOGRAPHIC DATA 



Variable (Year) 


Source 
Code ' 


North Carolina 


U.S. (Average) 


Rank 


Estimated Population (1973) 


1-3 


5,273,000 


209,851 ,000 


12 


■Median Years of Schooling Completed 
by the Population 25 Years Old 
and Older fl970) 


2 


10 6 


n2 1) 


AH 

to 


Median Famllv Income f1970) 


2 


$7,770 


f$9.867) 


An 


Percent of Population that is Black 


2 


22 2 


fll 11 


0 




2 , 


4fi 79R 




90 


Pnniil At4nn Ppr ^niiArA Hi 1a MQTO) 
rupuiai»iuii rci ov|Ucii c ni ic v 


2 


lOd 


fS7l 


17 


Percent of Population Classified 
Rural f 19701 


2*4 


55.0 


f26.5) 


c 
9 


Median Age of Population (1970) 


2 


26.6 


'(28.3) 


• 15 , 


Per-Capita Personal Income (1972) 


1-5 


$3,799 


($4,492) 


34 


Average Household Effective Buying 
Income (1972) 


1-6 


$7,441 


($8,605) 


39 


Percent of Household with Cash 
Incomes Under $3,000 (1972) 


1-6 


^19.4 


(15.4) 


12 


Percent of Housing Lacking Some or 
• All Plumbing Facilities (1970) 


2 


13.9 


(5.5) 


7 


Percent of Housing with 1.01 or More 
Persons Per Room (1970) 


,2 


10.0 


(8.0) 


13 


Percent of Homes with Telephone 
Available (1970) / 


2 


77.5 


(87.3) 


45 


^"fAfo AnH 1 nr;t1 T;iy PAV^niiA Per 

$1,000 of Personal Income During 
1971-7Z 


3-1 


$111.17 


($126.94) 


37 


pAv«r0n^;in0 P^r^on^ 9^ Ymt^ 
rcruciiLayc UT r ci duiid icaid 

and Over with Less Than 5 Years 
of School Completed (1970) 


2 


10.0 


(5.5) 


7 


Percentage of Personjs 25 Years and 
Over With 4 Years of High School 
or More (1970) 


2 


38.5 


(52^3)^ 


48 


Per-Capita State and Local Tax 
Revenue During 1971-72 


3-1 


$376.58 


($522.49) 


43 


Per-Capita. Direct General Expendi- 
tures of State and Local 
Governments During 1971-72 


3-1 


$563.45 


($801.38) 


49 



ERIC 



vdriduic \iXi<x* / 


Source 
Code 


North Carolina 


U.S. (Average) 


Rank 


Amount of Expenditures for Local 
Schools by State and Local 

Personal Income During 1971-72 


3-1 


$44.98 


A 

($53.27) 


46 


' State and Local Government * 

TvnAnH^ f ttv^oc -for All Public 

Education As a Percent of Total 
General Expenditures in 1971-72 


3-1 


42.7 


(38.9) 


15 


Per-Capita Total Expenditures of^^' 
State and Local Governments for 
all Education During 1971-72 


1-1 


$240.41 


($311.60) 


45 


Per-Capita State and Local Govern- 
mental Expenditures for Higher 
Education (includes community 
colleges) During 1971-72 


1-1 


$76.80 






Per-rCapita State and Local Expendi- 
tures for Local (public) Schools 
(Including Capital Outlay) 
During 1971-72 


1-1 


$152.37 


($219.27) 


47 


Revised Current Expenditure Per 
Rupil in Average Daily 
Membership:'^1971-72 

^ 1972-73 

(Estimate) 1973-74 


1-2 
1-2 
1-2 


696 
765 
846 


906 
968 
1048 


36 
37 
33 



Source Code : 

First Digit is Main Scource 

1 = National Education Association 

Ranking of the States* 1974 

2 = Bureau of the Census, County 

artiCity Data Book, 1972 

3 = Tax Research Division, North 

Carolina Department of Revenue, 
"Ranking of North Carolina Among 
the 50 States With Respect to 
Revenues and Expenditures, 
March, 1974" 



Second Digit is Primary Source 

1 = Bureau of Census, Governmental 

Finances in 1971-72 

2 =• NEA, Estimates of School 

Statistics 

3 - Bureau of Census, Estimates 

of the Population of the 
States 

4 = Bureau of Census, General 

Social and Economic 
Characteristics 

5 = "Survey of Current Business" 

6 = "Sales Management" 



